In an open computing environment, most software applications are usually available on a wide variety of hardware platforms. Generally, software applications written using open computing standards such as C, C++ and JAVA can be ported to several different hardware platforms with very little effort. In fact, applications written entirely in the architecturally neutral programming language JAVA require no actual porting and are categorized uniquely as "Write Once, Run Anywhere".sup.1. Increased portability of software applications means that computer hardware manufacturers can no longer leverage computer hardware sales on the exclusive availability of software applications. Consequently, a great number of computer manufacturers base their computer sales purely on computer hardware performance. In the end, these computer manufacturers compete to offer more processing power at increasingly lower costs. FNT 1. Write Once, Run Anywhere is a trademark of Sun Microsystems, Inc. in the United States and other countries. Write ONCE, Run Anywhere est une marque de fabrique ou une marque deposee de Sun Microsystems, Inc. aux Etats-Unis et dans d'autres pays.
Users generally compare benchmark results, such as SPECint or SPECfp, to determine which computing platform offers the best price-performance. In most cases, the computer manufacturers compile these benchmarks, execute them on their computers, and publish the results for each of their respective computing hardware platforms. Often, users will select a particular hardware platform based entirely on the performance results of these benchmarks. Open computing standards enable users to mix and match the best price-performance hardware solution in each performance class to create a robust and economical heterogeneous network.
The importance of these benchmarks is underscored by the amount of research and development spent on optimizing compilers. Many computer manufacturers increase the performance ratings on their product line by improving the optimizer in a compiler rather than improving the underlying hardware platform. Benchmarks run through the optimizer run faster on the same hardware platforms by streamlining the executable code and reducing the total number of clock-cycles per instruction (CPI) required to execute the benchmark.
Unfortunately, the user may discover after purchasing a particular hardware platform that the benchmark results do not translate into the higher overall expected performance characteristics. In some cases, a lower performing hardware solution may appear to have a better execution profile because of very sophisticated optimization techniques. Current benchmarking techniques do not enable the user to distinguish between highly optimized benchmarks and truly high performing hardware platforms.
What is needed is a reliable technique for users and others to generate executables such as benchmarks which can execute in a heterogeneous computing environment. Preferably, these benchmarks are substantially similar and can be used to provide dependable performance information on the different hardware platforms they are executed on.